Device and a method for annotating content

ABSTRACT

A device and a method for annotating content is provided. The device may comprise a means to analyse the content ( 150 ) and generate a first output ( 205 ) based upon the analysis of the content, a means to generate an annotation request ( 230 ), a means to communicate ( 130 ), the means to communicate being adapted to distribute the annotation request to at least one other device and to receive an annotation result from the at least one other device and a means to combine ( 240 ) the first output and the annotation result to provide an improved output. By combining multiple independent results of content analysis algorithms from different devices, possibly based upon background different knowledge a higher quality result is provided for the annotation.

FIELD OF THE INVENTION

The invention relates to a device for annotating content.

The invention further relates to a method for annotating content.

The invention further relates to a program element.

The invention further relates to a computer-readable medium.

BACKGROUND OF THE INVENTION

There is a trend of increasing electronics miniaturization, leading tothe development of devices that may have more processing power, allowingthem to become smarter. Furthermore, the trend of further technologyintegration will allow devices to integrate more and more technologiessuch as wireless networking and sensor capabilities into affordableproducts. The combination of these two trends will allow devices tobecome smart device, context aware and intelligently interacting withother such devices in a network (ad-hoc, fixed or otherwise). Suchdevices can be portable as well as stationary devices. Portable andstationary device makers tend to differentiate their device products, inboth form and function, from other such products in the market. Thiswill leave the user with even more such devices at home and on-the-move.

One use of such powerful devices is for the storage and/or rendering ofpersonal content. In using such devices metadata of personal content,such as own created photos and videos, is important to users to be ableto, for instance, easily organize, browse and find back their content.To manually annotate such content is a very laborious task for users.This is especially so since the amount of content produced, bothcommercially and personally, is ever increasing. Therefore, it isbecoming virtually impossible to properly annotate all newly createdcontent. Solutions are required that alleviate the users from sucharduous tasks and enable them to start enjoying the content.

Whilst a lot of solutions, using content analysis or otherwise, arebeing developed for the purpose of helping the user to annotate contentautomatically. None are satisfactory. When considering personal contentthe following types of metadata are generally found to be important:

Why was the content created? What is the “event”, e.g. Summer holidays

Who appears in the created picture or video? E.g. my wife

When was the content created? E.g. July, Summer

Where was the content created? E.g. In Italy

Further, types of metadata related to concepts and object present in thecontent, such as “happy”, “beach” and “tree” can also be of importanceto the user.

Concerning the recognition of who appears in photos and videos a lot ofliterature is available; see for instance Marc Davis, Michael Smith,John Canny, Nathan Good, Simon King, and Rajkumar Janakiraman, “TowardsContext-Aware Face Recognition,” Proceedings of 13th Annual ACMInternational Conference on Multimedia (MM 2005) in Singapore, ACMPress, 483-486, 2005. This article specifically targets context-awareface recognition in personal photos created using mobile phones. Afurther example is provided in Ara V. Nefian, Monson H. Hayes III, 1999,“Face recognition using an Embedded HMM”, which is a face recognitionmethod.

To determine where a content item was created, at creation time, it iswidely known that a Global Positioning System (GPS) can be used.Further, there are also systems developed that try to analyse thecontent created, to infer where the place is captured by the content.For instance, in Risto Sarvas, Erick Herrarte, Anita Wilhelm, and MarcDavis, “Metadata Creation System for Mobile Images,” Proceedings of theSecond International Conference on Mobile Systems, Applications, andServices (MobiSys2004) in Boston, Mass., ACM Press, 36-48, 2004, acreated image may be uploaded to a server to be compared with otherimages. From such an analysis it can be derived, for instance, that animage was taken at the “Campanile” tower on the UC Berkeley campus, USA.

Furthermore, there also many efforts to detect concepts and objects, seefor instance, Erik Murphy-Chutorianl, Sarah Aboutalib, Jochen Triesch,“Analysis of a Biologically-Inspired System for Real-time ObjectRecognition”, Cognitive Science Online, Vol. 3.2, pp. 1-14, 2005 and I.Cohen, N. Sebe, A. Garg, M. S. Lew, T. S. Huang, “Facial ExpressionRecognition from Video Sequences”, IEEE International Conference onMultimedia and Expo (ICME'02), vol II, pp. 121-124, Lausanne,Switzerland, August 2002.

However, even given all of the work being done in content analysis ithas been found that content analysis cannot provide 100% accurateannotation results. Whilst there are also efforts to incorporate userfeedback and learning algorithms it remains an issue that a user will berequired to provide significant amounts of feedback.

The inventors recognising this problem devised the present invention.

BRIEF SUMMARY OF THE INVENTION

The present invention seeks to improve the quality of annotation ofcontent.

Accordingly, there is provided, in a first aspect of the presentinvention, a device for annotating content, the device comprising ameans to analyse the content and generate a first output based upon theanalysis of the content, a means to generate an annotation request, ameans to communicate, the means to communicate being adapted todistribute the annotation request to at least one other device and toreceive an annotation result from the at least one other device and ameans to combine the first output and the annotation result to providean improved output.

The device may analyse the content locally using a content analyser,however, the results will, of course, be no better than the bestalgorithms available. However, by requesting at least one further deviceto also perform content analysis, a second independent annotation resultis determined. By combining the results of content analysis performedlocally on the device with the independent results of content analysisperformed and returned by at least one further device the individual andindependent results may be combined to provide a combined and improvedoutput.

According to a second aspect of the invention a method for annotatingcontent is provided, the method comprising the method steps of analysingthe content and generating a first output based upon the analysis of thecontent, generating an annotation request, communicating the annotationrequest to at least one other device and receiving an annotation resultfrom the at least one other device and combining the first output andthe annotation result to provide an improved output.

According to a third aspect of the invention a system for annotatingcontent is provided, the system comprising a plurality of devices forannotating content, each device of the plurality of devices comprising ameans to analyse the content and generate a first output based upon theanalysis of the content, a means to generate an annotation request, ameans to communicate, the means to communicate being adapted todistribute the annotation request to at least one other device and toreceive an annotation result from the at least one other device and ameans to combine the first output and the annotation result to providean improved output.

According to a fourth aspect of the invention a program element directlyloadable into the memory of a programmable device is provided,comprising software code portions for performing, when said programelement is run on the device, the method steps of analysing content tobe annotated and generating a first output based upon the analysis ofthe content, generating an annotation request, communicating theannotation request to at least one other device and receiving anannotation result from the at least one other device and combining thefirst output and the annotation result to provide an improved output.

According to a fifth aspect of the invention a computer-readable mediumdirectly loadable into the memory of a programmable device is provided,comprising software code portions for performing, when said codeportions are run on the device, the method steps of analysing content tobe annotated and generating a first output based upon the analysis ofthe content, generating an annotation request, communicating theannotation request to at least one other device and receiving anannotation result from the at least one other device and combining thefirst output and the annotation result to provide an improved output.

In one embodiment a user interface may be provided, the user interfacemay be adapted to present the annotation request to a user and toreceive feedback on the annotation request from the user and a means tocombine further combines the feedback from the user to provide theimproved output. This provides a fall back scenario for situations whenthe improvement in the quality of the annotation provided by theimproved output is still not judged to be sufficient.

In a further embodiment a means to communicate may be adapted to receiveexternal annotation requests from a further device and to transmit thefeedback from the user to the further device. This provides a device tooffer a service to further devices in a network to request and provideuser feedback even though the further devices may not have suitable userinterface capabilities to request feedback from the user themselves.

In another embodiment an annotation centre manager may be provided, theannotation centre manager may comprise a queue comprising a plurality ofoutstanding annotation requests for the user and a means to select aselected outstanding annotation request from the queue of outstandingannotation requests wherein the annotation centre manager iscommunicatively coupled to a user interface to communicate the selectedoutstanding annotation request to the user interface and to receive thefeedback from the user interface. This allows annotation requests to becollected and presented to a user at a suitable moment.

In yet another embodiment an annotation centre manager may furthercomprise a means to determine obsolete outstanding annotation requestswhich are outstanding annotation requests made obsolete by the feedbackfrom the user and a means to delete the obsolete outstanding annotationrequests from the queue and wherein a means to communicate is furtheradapted to communicate the feedback to originators of the obsoleteoutstanding annotation requests. This allows annotation requests thatwere queued to be analysed taking into account the feedback from theuser and annotation requests which may have become obsolete based on thefeedback from the user may be removed from the queue and communicatedback to the originator of the obsolete annotation requests. This alsoreduces the amount of user interaction required, by removing the need toanswer obsolete annotation requests.

In an embodiment a means to communicate may be further adapted toreceive a message identifying a further device as an annotation centremanager, transmit external annotation requests to the annotation centremanager identified and to receive external feedback from a user from theannotation centre manager identified and a means to combine may furthercombine the external feedback from the user to provide the improvedoutput. This allows a further device to present and receive userfeedback on behalf of the device and this is useful when the furtherdevice has more powerful user interface capabilities.

In another embodiment a means to detect that the user is in the vicinityof the device may be provided along with a means to identify the deviceas an annotation centre manager in a network of devices based upondetection that the user is in the vicinity of the device and a means tocommunicate may be further adapted to receive external annotationrequests from the network of devices. This allows annotation requests tobe directed to a local device to the user. This also allows feedback tobe gathered from the user in a fast and efficient manner.

In another embodiment a means for storing the improved output may beprovided. Storing the improved output allows the metadata relating tothe content to be permanently of an improved quality.

In a further embodiment a means to combine may further comprise aconfidence level determination unit, the confidence level determinationunit may be adapted to determine a first confidence level of the firstoutput, the first confidence level indicating the confidence that thefirst output is correct, determine a second confidence level of theannotation result, the second confidence level indicating the confidencethat the annotation result is correct and wherein a means to combine maybe adapted to further combine the first output and the annotation resultto provide an improved output based upon the first confidence level andthe second confidence level. This allows a confidence level to bedetermined for each result of content analysis, even in situationwherein the content analysis algorithm cannot provide a confidence levelitself.

In a further embodiment a device according to the invention may berealized as at least one of the group consisting of a Set-Top-Boxdevice, a digital video recording device, a network-enabled device, aconditional access system, a portable audio player, a portable videoplayer, a mobile phone, a DVD player, a CD player, a hard disk basedmedia player, an Internet radio device, a computer, a television, apublic entertainment device, a memory stick and an MP3 player. However,these applications are only exemplary.

In another embodiment an annotation request may be presented to a userand feedback on the annotation request may be received from the user anda method step of combining may further combine the feedback from theuser to provide the improved output. This provides a fall back scenariofor situations when the improvement in the quality of the annotationprovided by the improved output is still not judged to be sufficient.

In an embodiment external annotation requests may be received from afurther device and feedback from the user may be transmitted to thefurther device. This provides a service to further devices in a networkto request and provide user feedback even though the further devices maynot have suitable user interface capabilities to request feedback fromthe user themselves.

The data processing required according to the invention can be realizedby a computer program, that is to say by software, or by using one ormore special electronic optimization circuits, that is to say inhardware, or in hybrid form, that is to say by means of softwarecomponents and hardware components. The data processing may also beprovided by an external service running on a server.

The aspects defined above and further aspects of the invention areapparent from the examples of embodiment to be described hereinafter andare explained with reference to these examples of embodiment.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described in more detail hereinafter withreference to examples of embodiment but to which the invention is notlimited.

FIG. 1 illustrates a system diagram of a device according to anembodiment of the invention;

FIG. 2 illustrates in more detail a device according to an embodiment ofthe invention;

FIG. 3 illustrates in an embodiment of the invention wherein aconfidence level in the results of the content analysis is determined;

FIG. 4 illustrates in an embodiment of the invention wherein feedbackfrom a user is requested based upon a confidence level in the results ofthe content analysis;

FIG. 5 illustrates in an embodiment of the invention wherein annotationis requested from a further device based upon a confidence level in theresults of the content analysis;

FIG. 6 illustrates in an embodiment of the invention wherein annotationrequests are queued and obsolete annotation requests may be deleted fromthe queue based upon feedback from a user;

FIG. 7 illustrates in an embodiment of the invention wherein theproximity of a user to the device is detected such that the device mayinform further devices that it may receive annotation requests requiringfeedback from a user;

FIG. 8 illustrates a flowchart used in an embodiment of the invention;

FIG. 9 illustrates a second flowchart used in an embodiment of theinvention; and

FIG. 10 illustrates a third flowchart used in an embodiment of theinvention.

The Figures are schematically drawn and not true to scale, and theidentical reference numerals in different Figures refer to correspondingelements. It will be clear for those skilled in the art, thatalternative but equivalent embodiments of the invention are possiblewithout deviating from the true inventive concept, and that the scope ofthe invention will be limited by the claims only.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 illustrates an embodiment of the invention. A device 100 isprovided which may reside in a network 180. The network 180 may be alocale network or the Internet. A further device 165 may also reside inthe network. The device 100 may be an autonomous device, enabling device100 to work without explicit control of other devices and to control whoor what is accessing and using the functionality and data of the device100. This is of special importance to portable devices that need to workin different environments, some of which may be outside the users own(home/private) network. The device 100 may comprise a processor 120 forexecuting machine-readable code as is commonly applied in present daydevices. The device 100 may also comprise a bus 140 for interconnectingsystem components within the device 100. For example, the processor 120may use a memory 110 to run code and/or store data. The device may alsocomprise a means to communicate, such as network interface 130, allowingthe device 100 to communicate with the further device 165 or otherdevices.

The network interface 130 may be a network interface such as a wiredEthernet interface or it may also be wireless in nature. For example,the network interface 130 may be a WiFi, Bluetooth, mobile phone orother suitable network interface. A display 160 may be provided forrendering content to a user 192. The user 192 may interact with thedevice 100 suing a remote control 191 and a user interface 190. The userinterface 190 may of course also make use of the display 160. Otherforms of user interaction, beyond a remote control may also be suitable,such as a keyboard, a touch screen, a mouse or other point device. Astorage means 170 may also be provided for storing content, metadata andintermediate and/or final results of any processing steps. The storagemeans may be volatile or non-volatile and may be for example, SRAM, DRAMor FLASH memory, a hard disk drive or an optical drive. It is to benoted the content may be accessed from the storage means 170, butequally possible is that the content be accessed via the networkinterface 130. In such a case the storage means 170 is entirelyoptional.

A content analyser 150 may be provided to analyse content accessible bythe device, either locally or via the network 180. Analysis of thecontent may provide more metadata or metadata of an improved quality.This is especially relevant when considering personal content where thefollowing types of metadata are generally found to be important:

Why was the content created? What is the “event”, e.g. Summer holidays

Who appears in the created picture or video ? E.g. my wife

When was the content created? E.g. July, Summer

Where was the content created? E.g. In Italy

The user 192 also values metadata relating to higher-level concepts andobjects contained within the content, for example, the concepts of“happy”, “beach” and “tree” have been found to be of importance to theuser. As already noted when concerning the recognition of who appears inphotos and videos a lot of literature is available. For example,“Towards Context-Aware Face Recognition,” Proceedings of 13th Annual ACMInternational Conference on Multimedia (MM 2005) in Singapore, ACMPress, 483-486, 2005 specifically targets context-aware face recognitionin personal photos created using mobile phones. A further example isprovided in Ara V. Nefian, Monson H. Hayes III, 1999, “Face recognitionusing an Embedded HMM”, which is a face recognition method.

Other important metadata aspects are related to determining where acontent item was created, at creation time, it is widely known that aGlobal Positioning System (GPS) can be used. Further, there are alsosystems developed that try to analyse the content created, to inferwhere the place is captured by the content. For instance, in “MetadataCreation System for Mobile Images,” Proceedings of the SecondInternational Conference on Mobile Systems, Applications, and Services(MobiSys2004) in Boston, Mass., ACM Press, 36-48, 2004, a created imagemay be uploaded to a server to be compared with other images. From suchan analysis it can be derived, for instance, that an image was taken atthe “Campanile” tower on the UC Berkeley campus, USA.

Furthermore, there also many efforts to detect concepts and objects, seefor instance, Erik Murphy-Chutorianl, Sarah Aboutalib, Jochen Triesch,“Analysis of a Biologically-Inspired System for Real-time ObjectRecognition”, Cognitive Science Online, Vol. 3.2, pp. 1-14, 2005 and I.Cohen, N. Sebe, A. Garg, M. S. Lew, T. S. Huang, “Facial ExpressionRecognition from Video Sequences”, IEEE International Conference onMultimedia and Expo (ICME'02), vol II, pp. 121-124, Lausanne,Switzerland, August 2002.

All of these forms of content analysis are useful, however, they are notreliable enough to work on their own. Therefore there are also effortsto incorporate user feedback. This feedback can be used to makeannotations where the content analysis method results have lowconfidence and to improve the content analysis algorithms (onlinelearning). For example, the “Semantic Learning and Analysis ofMultimedia”, SLAM project, IBM Research,http://www.research.ibm.com/slam/. The known solutions mostly focus onhow to use feedback for online learning of content analysis algorithms.

A completely different way for users to get their content annotated isthrough “collaborative tagging”, for example in Scott A. Golder andBernardo A. Huberman, “The Structure of Collaborative Tagging Systems”,to appear in the Journal of Information Science (2006), InformationDynamics Lab, HP Labs. In this solution users may add annotations to anycontent uploaded to a website.

In the following annotation means not only the creation of metadata butalso the creation of user added information. Annotations may beoptionally be stored to assist in future browsing or search actions.

It remains the case that user feedback is only reliable for the explicitquestions posed to a user and any inferences taken from responses of auser will not be completely reliable. Furthermore, due to the widevariety of processing platforms, i.e. devices, and varying capabilitiesof devices the extent to which the results of any single contentanalysis algorithm can be trusted will be limited.

In FIG. 2 an embodiment is shown in more detail that improves thequality of output from content analysis. Content 210 to be annotated ismade available to the device 100. The content 210 may be photos, audio,video or some other data form. The content analyser 150 analyses thecontent 210 using a known algorithm, for example, face detectingalgorithm. The content analyser 150 generates a first output 205 withthe results of the content analysis step. The first output 205 maycomprise only annotations or annotations and a measure of the confidencein the annotations. The actual annotations and any measure of theconfidence are specific to each any every content analysis algorithm, sosome form of standardisation of, especially, the measure of theconfidence may be useful. The first output 205 may comprise, forexample, the number of faces, who the faces represent, etc. Many othercontent analysis algorithms are also possible that focus on videofeatures, such as colour etc, or audio features, such as volume, tempoetc.

The first output 205 is communicated to an annotation manager 220. Theannotation manager 220 may comprises an annotation request generator 230and a combiner 240. The annotation request generator 230 receives thefirst output 205 and generates an annotation request 215 based upon thefirst output 205. Optionally, the annotation request generator 230 mayonly provide a link to the content 210 in the annotation request 215such that the further device 165 may analyse the content 210 fromscratch. The annotation request 215 is communicated internally to thenetwork interface 130, from where it is distributed within the network180 to other devices, such as, further device 165.

In this example, the further device 165 analyses the content 210according to the annotation request 215. The annotation request 215 maybe a general request to analyse the content 210 from scratch, or it maybe a specific request to analyses a certain aspect of the content 210.For example, a specific request may a request to analyse only the facesdetected in a photo. After performing the analysis, the further device165 will reply to the annotation request 215 with an annotation result225. The independent annotation result may be performed by a differentcontent analysis algorithm or be analysed based upon differentbackground knowledge. The annotation result 225 may comprise only theannotation metadata or optionally also a measure of the confidence inthe annotation metadata.

The device 100 receives the annotation result 225 from the furtherdevice 165 via the network interface 130. The combiner 240 receives theannotation result 225 and combines it with the first output 205. Thecombiner 240 analyses the two independent results and judges the qualityor confidence in one or both results to provide an improved output 235.The improved output 235 may be any known combination or function of thefirst output 205 and the annotation result 225.

In another embodiment as shown in FIG. 3 the annotation manager 220 maycomprise a confidence level determination unit 310. The confidence leveldetermination unit 310 is shown to intercept the first output 205 fromthe content analyser 150. For clarity it is noted that the contentanalyser 150 may be a hardware component or a software module running onprocessor 120. If the content analyser 150 does not provide any measureof confidence in the first output 205, the confidence leveldetermination unit 310 may estimate the confidence level usinghistorical information. Based upon the level of confidence in the firstoutput 205 the confidence level determination unit 310 may decide totrigger the annotation request generator 230 to generate the annotationrequest 215 or pass the first output 205 directly to the improved output235 via switch 320. Should the path be chosen making use of theannotation request 215 then the annotation request 215 may be processednormally and communicated to the improved output 235 via the combiner240 and switch 320, when set in the lower position.

In FIG. 4 a further embodiment is shown. In FIG. 4 the annotationmanager 220 may comprise an annotation request receiver 410 capable ofreceiving annotation requests 215 from devices on the network 180. Theannotation request receiver 410 may be capable of initiating the contentanalysis of the content 210, using the content analyser 150. The content210 may be remotely located, as shown in FIG. 4. The content 210 mayalso be stored on the further device 165. The annotation manager 220 mayfurther comprise an annotation question generator 420. The annotationquestion generator 420 may be communicatively coupled to the confidencelevel determination unit 310 and be triggered by the result of theanalysis performed by the confidence level determination unit 310 togenerate a question for the user 192. The display 160 and the userinterface 190 may be used for this purpose. The user 192 may providefeedback 415. Based upon the feedback 415 the annotation questiongenerator 420 generates an annotation result 225 of a high confidencelevel. The annotation result 225 may be communicated back to anoriginator of the annotation request 225. This communication may beperformed by the network interface 130, though this is not shown in FIG.4.

In FIG. 5 an embodiment is shown wherein an annotation question may becommunicated via network interface 130 to a remote device 510 comprisingan annotation centre manager 520. The annotation centre manager 520displays the annotation question to the user in a similar manner asdescribed in FIG. 4. The feedback 415 is communicated from theannotation centre manager 520 of the remote device 510 via the network180 to the annotation question generator 420 of the device 100. Theannotation result 225 is then generated by the annotation questiongenerator 420. Again the annotation result 225 may be communicated toother devices via the network interface 130.

In FIG. 6 an embodiment is shown where device 100 may act as anannotation centre manager 520 in the network 180. The annotation centremanager 520 may comprise an annotation request receiver 410 and anannotation request queue 620. This allows annotation requests to becollected until the user 192 is ready to give feedback. The annotationrequest queue 620 may trigger the user interface 190 to directly requestfeedback from the user 192. Alternatively, the annotation request queue620 may trigger the content analyser 150 to analyse the content 210.Based upon the feedback 415 from the user 192 an obsolete requestsdetermination unit 630 may determine which requests in the annotationrequest queue 620 are now obsolete taking into account the feedback 415provided by the user 192. For example, if after the feedback 415 hasbeen analysed that the face of “John” may now be recognised with a highconfidence. Any other annotation requests relating to the face of “John”do not need to be forwarded to the user 192. Therefore, the workloadrequired from the user 192 is reduced.

In FIG. 7 another embodiment of device 100 operating as an annotationcentre manager 520 is shown. The annotation centre manager 520 may alsocomprise a user proximity detection unit 710. The user proximitydetection unit 710 may use any known means to detect that the user 192is in close proximity to the device 100. For example, the user proximitydetection unit 710 may use a camera, a thermal detector etc. The userproximity detection unit 710 may also infer the proximity of the user192 by noting user interaction and the time elapsed since suchinteraction occurred. For example, if the user 192 is operating, or hasvery recently operated, the device 100 then the user 192 will probablybe within close proximity to the device 100. The user proximitydetection unit 710 may then inform other devices in the network 180, viaan indication 705, that it should become the current central annotationcentre manager and all annotation requests or questions requiring userfeedback should be directed to device 100.

In FIG. 8 a flowchart is shown indicating a method for implementing anembodiment of the invention on the processor 120 of device 100. In step800 content 210 is received, or at least made accessible. In step 810the content 210 is analysed locally using a content analysis algorithmand the first output 205 is produced. At step 820 annotation request 215is generated and at step 830 the annotation request 215 is communicatedto a further device 195. At step 840 annotation result 225 is receivedfrom the further device 195. At step 850 the first output 225 and theannotation result 225 are combined to provide an improved output 235.

In FIG. 9 a second flowchart is shown indicating a further method forimplementing an embodiment of the invention on the processor 120 ofdevice 100. After step 820 two parallel paths are possible. The firstpath encompasses method steps as described in the text of FIG. 8, namelysteps 830 and 840. The second path presents the annotation request 215as a question to the user 192 in step 910. The presentation to the user192 may be on a local device, for example, device 100, or the furtherdevice 165. The exact location of the user 192 is not of importance. Itis however important that the user 192 provides the feedback 415 in step920. In step 930 the first output 205, the annotation result 225 and thefeedback 415 are all combined to provide a very high quality annotationas an improved output 235.

In FIG. 10 a third flowchart is shown indicating a further method forimplementing an embodiment of the invention on the processor 120 ofdevice 100. In the embodiment of FIG. 10 at step 1010 externalannotation requests may be received from a remote device. Step 910 andstep 920 are identical to those steps described in FIG. 9, but operateon the external annotation request. Again, the presentation to the user192 may be on a local device, for example, device 100, or the furtherdevice 165. At step 1020 the feedback 415 may be transmitted back to theoriginator of the external annotation request. This allows annotationrequests to be transferred to a location close to the user 192.

In summary the invention discloses methods and devices for annotatingcontent. The device may comprise a means to analyse the content andgenerate a first output based upon the analysis of the content, a meansto generate an annotation request, a means to communicate, the means tocommunicate being adapted to distribute the annotation request to atleast one other device and to receive an annotation result from the atleast one other device and a means to combine the first output and theannotation result to provide an improved output. By combining multipleindependent results of content analysis algorithms from differentdevices, possibly based upon background different knowledge a higherquality result is provided for the annotation.

It should be noted that the above-mentioned embodiments illustraterather than limit the invention, and that those skilled in the art willbe capable of designing many alternative embodiments without departingfrom the scope of the invention as defined by the appended claims.Furthermore, any of the embodiments described comprise implicitfeatures, such as, an internal current supply, for example, a battery oran accumulator. In the claims, any reference signs placed in parenthesesshall not be construed as limiting the claims. The word “comprising” and“comprises”, and the like, does not exclude the presence of elements orsteps other than those listed in any claim or the specification as awhole. The singular reference of an element does not exclude the pluralreference of such elements and vice-versa. In a device claim enumeratingseveral means, several of these means may be embodied by one and thesame item of hardware. The mere fact that certain measures are recitedin mutually different dependent claims does not indicate that acombination of these measures cannot be used to advantage.

1. A device (100) for annotating content, the device comprising: a meansto analyse (150) the content and generate a first output (205) basedupon the analysis of the content; a means to generate (230) anannotation request (215); a means to communicate (130), the means tocommunicate being adapted to distribute the annotation request to atleast one other device (165) and to receive an annotation result (225)from the at least one other device; and a means to combine (240) thefirst output and the annotation result to provide an improved output(235).
 2. The device of claim 1 further comprising a user interfaceadapted to present the annotation request to a user and to receivefeedback on the annotation request from the user and wherein the meansto combine further combines the feedback from the user to provide theimproved output.
 3. The device of claim 2 wherein the means tocommunicate is further adapted to receive external annotation requestsfrom a further device and to transmit the feedback from the user to thefurther device.
 4. The device of claim 2 further comprising anannotation centre manager, the annotation centre manager comprising: aqueue comprising a plurality of outstanding annotation requests for theuser; and a means to select a selected outstanding annotation requestfrom the queue of outstanding annotation requests wherein the annotationcentre manager is communicatively coupled to the user interface tocommunicate the selected outstanding annotation request to the userinterface and to receive the feedback from the user interface.
 5. Thedevice of claim 4 wherein the annotation centre manager furthercomprises: a means to determine obsolete outstanding annotation requestswhich are outstanding annotation requests made obsolete by the feedbackfrom the user; and a means to delete the obsolete outstanding annotationrequests from the queue wherein the means to communicate is furtheradapted to communicate the feedback to originators of the obsoleteoutstanding annotation requests.
 6. The device of claim 1 wherein themeans to communicate is further adapted to: receive a messageidentifying a further device as an annotation centre manager; transmitexternal annotation requests to the annotation centre manageridentified; and to receive external feedback from a user from theannotation centre manager identified and wherein the means to combinefurther combines the external feedback from the user to provide theimproved output.
 7. The device of claim 3 further comprising: a means todetect that the user is in the vicinity of the device; a means toidentify the device as an annotation centre manager in a network ofdevices based upon detection that the user is in the vicinity of thedevice and wherein the means to communicate is further adapted toreceive external annotation requests from the network of devices.
 8. Thedevice of claim 1 further comprising a means for storing the improvedoutput.
 9. The device of claim 1 wherein the means to combine furthercomprises a confidence level determination unit, the confidence leveldetermination unit being adapted to: determine a first confidence levelof the first output, the first confidence level indicating theconfidence that the first output is correct; determine a secondconfidence level of the annotation result, the second confidence levelindicating the confidence that the annotation result is correct; andwherein the means to combine is further adapted to combine the firstoutput and the annotation result to provide an improved output basedupon the first confidence level and the second confidence level.
 10. Thedevice of claim 1 realized as at least one of the group consisting of: aSet-Top-Box device; a digital video recording device; a network-enableddevice; a conditional access system; a portable audio player; a portablevideo player; a mobile phone; a DVD player; a CD player; a hard diskbased media player; an Internet radio device; a computer; a television;a public entertainment device a memory stick; and an MP3 player.
 11. Amethod for annotating content, the method comprising the method stepsof: analysing the content and generating a first output based upon theanalysis of the content; generating an annotation request; communicatingthe annotation request to at least one other device and receiving anannotation result from the at least one other device; and combining thefirst output and the annotation result to provide an improved output.12. The method of claim 11 further comprising the method steps of:presenting the annotation request to a user and receiving feedback onthe annotation request from the user; and in the method step ofcombining further combining the feedback from the user to provide theimproved output.
 13. The method of claim 12 further comprising themethod steps of: receiving external annotation requests from a furtherdevice; and transmitting the feedback from the user to the furtherdevice.
 14. A system for annotating content, the system comprising aplurality of devices for annotating content, each device of theplurality of devices comprising: a means to analyse the content andgenerate a first output based upon the analysis of the content: a meansto generate an annotation request, a means to communicate, the means tocommunicate being adapted to distribute the annotation request to atleast one other device and to receive an annotation result from the atleast one other device; and a means to combine the first output and theannotation result to provide an improved output.
 15. A program elementdirectly loadable into the memory of a programmable device, comprisingsoftware code portions for performing, when said program element is runon the device, the method steps of: analysing content to be annotatedand generating a first output based upon the analysis of the content;generating an annotation request; communicating the annotation requestto at least one other device and receiving an annotation result from theat least one other device; and combining the first output and theannotation result to provide an improved output.
 16. A computer-readablemedium directly loadable into the memory of a programmable device,comprising software code portions for performing, when said codeportions are run on the device, the method steps of: analysing contentto be annotated and generating a first output based upon the analysis ofthe content; generating an annotation request; communicating theannotation request to at least one other device and receiving anannotation result from the at least one other device; and combining thefirst output and the annotation result to provide an improved output.